Stopping to think: reflections on the use of portfolios

Karen Barton and Fiona Westwood, Glasgow Graduate School of Law

This paper from Vocational Teachers Forum V describes a pilot project into the use of portfolios at Glasgow Graduate School of Law (GGSL). The pilot was rolled out to new trainees in five law firms and a number of Diploma in Legal Practice students from September 2005, and from 2006 was taken forward as an e-portfolio – see Getting started with e-portfolios: GGSL case study for details.

---

In many professions the use of portfolios is now commonplace as a route to qualification, revalidation or continuing practice. The benefits of portfolios for professional development include their capacity to look at technical development over time, to reflect on and demonstrate progress, and to support the integration of work-based learning and assessment.

For example, the Royal Institute of Chartered Surveyors’ Assessment of Professional Competence (APC) is a highly structured two year programme leading to membership of the Institute. Under supervision, trainees maintain daily diaries and monthly logbooks, undertake interim reviews and set professional development plans. Prior to a final assessment interview trainees must submit a portfolio that includes proforma reports, logs and a critical analysis of a project they have been involved with.

The medical profession has been required recently to look more closely at the issues of professional standards, lifelong learning and fitness to practice. In the wake of Shipman and the Bristol Royal Infirmary Inquiry the profession has shifted its emphasis towards public accountability, revalidation and continuing professional development, as opposed to a combination of an historical record of a doctor’s initial qualifications and a system that responds to complaints of poor practice. The General Medical Council (GMC) outlined this new approach to professionalism in their Good medical practice document published in 2001.

In response to this, in 2002 the Royal College of General Practitioners (RCGP) set out the standards for revalidation of GPs in Good medical practice for general practitioners. In Scotland a ‘revalidation folder’ has been introduced to facilitate and streamline the process of producing and organising the evidence required for appraisal and revalidation, structured under the GMC’s seven headings of good medical practice. Each has criterion statements and standards to be applied against these criteria, together with descriptions of valid evidence that can be used to demonstrate the standards are being met and various pro formas to help with organisation – for full details see the Appraisal & Revalidation section of RCGP Scotland’s website.

Higher education has also seen the introduction of processes to formalise reflection. At undergraduate level, as part of an increasing realisation that encouraging reflective practice and lifelong learning is a fundamental tenet of tertiary education, university students are gaining more experience with portfolios in individual modules and courses as well as through the introduction of personal development planning schemes.

The legal profession, however, is currently someway behind. While there are good examples of portfolios in use within some law subjects at undergraduate level – see for example the case studies in the UKCLE publication Developing reflective practice – is there a place for portfolios within the legal profession and professional qualification? Are there any specific issues about legal practice that we need to consider? Do portfolios provide an additional dimension to vocational legal education and training that is currently missing, or are they just more work and a passing fad?

These were some of the questions raised at a conference organised by the Law Society of Scotland in December 2004 to discuss developmental ideas for legal vocational training. In Scotland, qualification as a solicitor and entry to the legal profession requires an undergraduate Bachelor of Laws degree (LLB), followed by a professional training programme consisting of a one year, full time Diploma in Legal Practice course and two year traineeship. During the traineeship, activity logs are completed and individual quarterly reviews take place within the training firm. Trainees are also required to attend an accredited Professional Competence Course consisting of a core skills course and a number of electives. In a research study commissioned by the Law Society of Scotland into the usefulness of the Diploma in Legal Practice and presented at the conference (Westwood, 2003), it was reported that the perception among the profession was that none of this training is currently inter-linked.

Among other topics aimed at addressing this issue, the applicability of portfolios to the legal profession was discussed. The conference agreed that this particular area required further research. As a result, the Glasgow Graduate School of Law (GGSL) decided to set up a two year pilot project, inviting a small group of law firms who take on trainees on a regular and structured way to work with us to ensure that contributors gain directly from their involvement in the project. The pilot seeks to investigate the use of portfolios within the Scottish legal professional training framework and to determine whether portfolios have the potential to improve the development of trainees’ skills and knowledge, and to maximise employer investment in taking the time to train them.

Reflective practice: why?

The aim of the Diploma in Legal Practice is to equip students with the basic skills, knowledge and competencies required in order to practice as a lawyer. However, if we are serious about preparing law students for legal practice then the Diploma should not just focus on equipping them with a set of ‘legal skills’, but should also seek to develop values, attitudes and an ethical approach to professionalism (Moore, 1970; Paterson, 2000) In the words of Schon (1987), it should be about preparing them to survive the ‘swamp’.

The ‘swamp’ of legal practice requires lawyers to meet both their professional and commercial aspirations (Paterson, 1996; Westwood, 2001 & 2004). Increased consumerisation is resulting in clients who are suspicious of their legal advisors and more likely to complain about their services (see recent Annual Reports of the Scottish Legal Services Ombudsman). The current debate on the removal of self regulation highlights public dissatisfaction with the profession’s current system of reflection at the macro level. At the micro level, if the legal profession is to respond to these external pressures its members are required to reflect on and improve their current provision and to develop new services and products.

The study of law at undergraduate level tends to concentrate on the application of legal principles to specific subject areas, rather than focus on the vocational aspects of lawyering. As a result:

“In order to find points of contact between professionals’ education, knowledge and work, one must use other concepts of knowledge – such as the reflective type of knowledge – which can better explain how professional proficiency is embedded in practice itself, with the contextual conditions that are tied to specific situations.”

— (Svensson, 1990:69-70)

We therefore need to think about “intuition, insight and artistry” (Schon, 1983), and developing not just competent, but also reflective and ethical practitioners. And to do this we need to engage the students in experiential learning and reflective practice (McLoughlin & Luca, 2002).

At GGSL we already do this in a number of ways, for example through the use of simulations, role plays, elective projects, reflective reports, activity and personal logs, and discussion forums. Our virtual town of Ardcalloch, where students practise in firms and undertake a series of legal transactions over the course of the year, is a good example of successful experiential learning in practice, and the benefits of this learning environment have been widely disseminated (see Maharg, 2001; Maharg, 2004a & 2004b).

Within this environment we also try to encourage reflective practice, and we see this as being vital for a number of reasons, including that:

• it is important to the development of professionals because it enable us to learn from experience (Beatty, 1997)
• students need to develop meta-cognitive skills because these skills affect their ability to understand and make sense of experience and are essential to the process of reflection and working in situations of uncertainty (Svensson, 1990)
• reflective practitioners draw on intuition to do what feels right (Schon, 1983)
• asking students why they behaved or interpreted a situation in a particular way provides useful insights not only into how much and what they understand but also the extent to which they draw on intuition (Hinett, 2002)

Why portfolios?

There are several models of portfolios, depending on whether the emphasis is on showcase, process or product. In the sense we are using it here the key quality is that the portfolio should enable learning through a combination of experience and reflection on experience. The principal feature is that this form of portfolio is produced as part of a learning process. The process is important, and highlights the difference between doing and learning – we can do a lot and learn little. One of the results of the traditional assessment systems we tend to use is that they condition students into focusing on products and ignoring the process of standing back and reflecting that allows them to become more knowledgeable – “‘high ground’ learning alone will not enable competent practice” (Maughan & Webb, 2005:35).

A second feature is that portfolios contain direct evidence of work and learning. They include multiple examples of real, context rich work, plus evidence of an individual’s ability to make choices. As a result they make learning more understandable and directly relevant to the learner. Portfolios represent achievement and learning. They offer a look at development over time, an ability to add to and reflect on previous entries, and demonstrate progress towards a goal. They may also comprise an explicit demonstration that specified learning outcomes have been achieved.

In creating portfolios individuals go though an experiential cyclical of doing, collecting, selecting and reflecting. The end result is an extremely authentic learning process and form of assessment that:

• provides an opportunity for continuous assessment over the duration of a course, a professional development programme or even as an ongoing process
• supports the integration of learning and assessment, particularly if there is an element of work-based learning
• supports the demonstration and valid assessment of a wide range of personal and professional capabilities which are difficult to assess by other methods (for example, attitudes and professionalism)
• shows reflection on and analysis of evidence and learning, higher order levels of thinking and key features of professionalism that we want to develop, and help build links between theory and practice
• is more versatile than other assessment methods in that a variety of evidence can be used to demonstrate one or more outcomes have been achieved
• places responsibility for learning on the student/trainee
• moves students from describing the course in terms of subjects (for example, “I’m studying Professional Ethics or Private Client”) or assessments to be done (for example, “I’ve got an exam or an essay”) to describing learning and achievements (for example, “I’ve learned how to communicate with clients more effectively”)

These are among the advantages of portfolios highlighted by Baume (2001) in general and by Webb (2003) with particular reference to the study of law.

We therefore see the introduction of portfolios to the professional legal training regime as entirely consistent with our current approach, and also as a valuable tool to bring together all the elements of the Diploma course and subsequent traineeship into a ‘coherent tale of learning’ (Baume, 2001) for the student. Moreover, a key quality of portfolios is that not only can they enable learning for an individual, but they can demonstrate that learning effectively to multiple audiences, including assessors, employers, peers and the public. It is perhaps this feature of portfolios that is potentially the most powerful and persuasive argument in their favour.

Portfolio pilot plan

At the Law Society of Scotland conference in 2004 a potential model for a portfolio was proposed, spanning the three years of professional training and education and culminating in a final assessment of the portfolio itself. While delegates expressed some interest in the idea, there was clearly a need for further research. Following the conference a team at GGSL discussed the possibility of setting up a portfolio pilot project and devised a research project plan, running over two years and covering three phases:

1. Awareness raising and recruitment of firms.
2. Setting the parameters of the pilot in partnership.
3. Implementation and review of the scheme.

Phase 1: awareness raising

We wrote to 18 training firms that we worked with on the Professional Competence Course, outlining the pilot project and inviting them to a workshop on portfolios in professional practice. Nine firms attended the initial workshop. The Deputy Director of the Law Society of Scotland presented at the workshop, along with the President of Royal College of General Practitioners in Scotland, who explained the rationale behind the revalidation folder and demonstrated their approach. The firms discussed and agreed current issues with regard to trainee development and the feasibility of introducing portfolios as part of the training process. Following the workshop we wrote to the firms again asking them to confirm their willingness to be involved. Five agreed to take part.

A one to one meeting was arranged with each of the firms, where we asked them to identify their current trainee support mechanisms and the benefits they would like portfolios to achieve. All participating firms met together again for a feedback meeting to review the findings, consider sample processes and tools and agree the level of involvement and timescales. At the same time we devised an elective project as part of the Diploma course that would require students to construct their own portfolio.

Our role in the pilot was to act as coordinator to monitor and support participants, and in return to gain unique insight into the practical implementation of this new approach within training firms. The employers would benefit from sharing sample documents, working practices and experiences.

Pros and cons

The initial workshop asked the firms to discuss what they saw as the benefits of introducing portfolios. They identified consistency, validation of the assessment process and the linkage between the Diploma outcomes and professional practice as key advantages. The creation of a personal learning record that involved clients and colleagues and could help identify personal preferences and directions for trainees was seen as being especially valuable. In particular, they saw the portfolio as a useful tool to help them in the thorny area of the recruitment of trainees and in the selection and retention of them at the end of their training period.

However, they were also quick to express reservations in several areas, including the time cost involved in monitoring and interpretation. There were concerns about possible duplication with the current Law Society logbooks and a potential increase in the responsibility placed on firms. All of this was seen as crucial to gaining the buy-in of partners. Maintenance of standards, as well as general concerns about attitudes and a ‘fear factor’ within the firm, were also raised.

As far as the practicalities went, there was a lengthy debate about the structure of the portfolio itself. The concerns raised focused around who was responsible for the specification, as well as the flexibility of the structure itself. Would the portfolio be organised around learning outcomes, categories of work, a diary, or combinations of these? And, of course, client confidentiality and limits to the size of the portfolio were also highlighted as issues.

As a first step to towards addressing some of these issues we were aware that we needed to define, minimally, learning outcomes or headings for each stage, as well as tools or pro formas to be used (for example, learning logs, significant event analysis forms, client feedback forms, skills assessment tools for key areas). It was clear that the portfolio specification must be clear and simple and allow alternative models, but we also needed to look at the support mechanisms trainees would require. Guidance on the use of portfolios and examples would be necessary so that they were aware of what was expected. Trainees would require the help of mentors, but we also needed to support conversation and peer support within the firm. The use of forms and checklists might help avoid ‘empty box syndrome’, and regular reviews and feedback were crucial.

Phase 2: setting the parameters

Having debated the main issues as a group we then met with each firm individually to discuss the pilot in more detail and how it could be integrated into their current processes and procedures. As a direct response to the concerns they had raised as a group we felt it was important to identify their own style of approach and provide encouragement that the pilot would be able to adapt to them, rather than the other way round.

Our discussions with individual firms revealed a number of common support mechanisms currently in place for trainees. All firms used the Law Society log books and quarterly reviews, as would be expected, but they also engaged in one to one formal interviews with a senior member of the firm as well as informal chats and shadowing. All firms organised in-house training sessions which trainees attended. They also provided support with elective selection on the Professional Competence Course and required written handovers between seats from one trainee to the next.

In addition, some firms ran informal evenings or more formal in depth induction sessions with trainees. One firm had drawn up their own competency framework and skills matrix. Another had devised a system of formal allocation of scores for quarterly reviews to demonstrate improvement over the two years of the traineeship. Most firms reported they had mentors and a system of mentor training in place. One firm used assistants or associates as mentors in year 1 and partners in year 2 to reinforce the transition from ‘learning the job’ to ‘career development’. One to one discussions with mentors before trainees commenced their final seat were used in at least one firm to help trainees make the correct career choice.

Following these individual meetings we met together again as a group to consider what we had learned and agree a way forward. At this meeting we discussed the aims of the pilot, timescales, procedural issues and the outcomes that would allow us to measure the success of the pilot upon completion.

Measures of success: trainees and students

Primarily, we agreed, there must be some benefit to the participants themselves. Among these we would expect to see trainees and students:

• take ownership of their learning, identify and action areas for improvement
• provide evidence of work carried out and learning achieved over time
• complete self development reports linking reflection, learning and ownership
• demonstrate more proactive input in completion of the review forms
• demonstrate a tangible improvement in professional and personal skills
• demonstrate improved learning from courses on the Professional Competence Course
• succeed in securing jobs after completion of traineeship
• recognise the investment being made by the firm in their own development
• own a valuable resource to fall back on to reflect learning and tools to tackle learning situations

Measures of success: firms

For the firms taking part in the pilot there must also be some tangible benefits. Essentially, the benefits identified concerned improvements in monitoring, planning and decision making regarding trainee development, plus improvements in the firm’s ability to:

• obtain good quality evidence of learning during traineeship
• place the onus for learning on individual trainees
• forward plan for trainee development in a comprehensive and consolidated way
• achieve more consistency between trainees’ individual abilities
• achieve better objective structure of assessment between mentor and trainee
• deal more effectively with any trainee’s underperformance
• demonstrate within the firm the investment being made in trainee development
• obtain objective evidence of future employability of trainees

Measures of success: GGSL

While we were interested in these measures of success from a practical point of view, we also had a number of our own motivations and measures to keep in mind. These included information and suggestions to:

• assist us on the areas of skills development covered by the Diploma in Legal Practice and Professional Competence Course
• inform us about the needs of employers as they impact on the employability of our students
• feed back to the Law Society of Scotland about the current systems of trainee development
• improve the linkages between the Diploma in Legal Practice and the Professional Competence Course
• build on the existing relationships between ourselves and the training firms
  There were also other measures that were more esoteric and difficult to quantify, but important none the less, and required us to refocus on the underpinning theories we hoped to test.

If we accept that reflection can be used as a way of helping individuals to take responsibility for their own learning and to identify ways to advance their practice and professional conduct then we must attempt to measure how successful the portfolio approach is in achieving this. From the evidence and reflections they produced, we would hope to find that trainees had developed the perspective to ask why, given a set of circumstances, lawyers should ‘do’ something in a particular way. We would expect to see trainees express themselves clearly, choose appropriate courses of action, evaluate decisions and juggle clients and problems more effectively.

One of our main aims is to help trainees develop ways of discovering and integrating their personal values into their professional practice, by promoting the idea that reflection is central to professional conduct. As law is an “ethically saturated arena” students and trainees need tools to reflect on their own values and to equip them to reach decisions (LeBrun & Johnstone, 1994:165). As lifelong learners, we should encourage their “resilience, resourcefulness and reflection” (Claxton, 1999:180). We therefore need to investigate how well portfolios provide students and trainees with coping mechanisms and an ability to cope with constructive feedback.

Feedback from firms

Each of the representatives of the firms responded positively to our group discussions and suggestions, seeing the relevance and application of what was being proposed. Enthusiasm from within the partner firms was not quite so high. A few of the firms had useful feedback to offer as well as discerning comments to make. Some questioned the practicality of what was being proposed, calling it “too esoteric and impractical to implement”. There were reservations about the idea of reflection, suggesting that it was too difficult, or did not sit comfortably within the culture of the firm, and that “a list of practical tasks would be better than a list of evidence”. An element of numerical scoring to help trainees to know where they were was suggested. The issue of confidentiality of evidence was raised again, and doubt expressed whether trainees would submit work as evidence that they knew to be below par. The level of support and amount of time needed from mentors in order to get the trainees through the process was seen as daunting, and reservations expressed that it would be asking too much of trainees to complete the trainee logbook and the portfolio alongside it.

Phase 3: implementation

Taking these perceptive observations on board we proposed a three level model of participation in the pilot project, to address individual firms’ reservations and suit their existing practices.

• Level 1 involves the trainee in a simple three stage reflection prior to each quarterly review summarising:
  • what went well (with supporting evidence)
  • what needs to be improved (with supporting evidence)
  • what is planned to do as a result
• Level 2 requires the firm to adapt existing processes to include more formal collation of evidence and written personal reflection by the trainee
• Level 3 involves the firm in introducing a more advanced portfolio format using some of the tools and templates that we developed

Each participating firm was allowed to select the level appropriate to its preferred style of approach.

At the same time 35 volunteers were recruited for the project on the Diploma course. This number included all the students who had already secured a traineeship with one of the partner training firms, plus more than 20 others who elected to participate. The group was representative of the demographic profile of the whole Diploma cohort, in terms of age range, gender and route of entry to the course.

An initial workshop was held with this Diploma pilot group where the concept of experiential learning and reflection was introduced. A number of reflective exercises were carried out by the students individually, including a ‘values’ exercise, practising reflection and reflective writing. Students were provided with tools such as personal logs, structured reflection and critical incident analysis pro formas. Around six weeks after the workshop the students were asked to complete an interim personal statement and invited to a review meeting with their mentor, where the completion of personal logs and the collation of evidence were discussed.

Initial reflections

Although only six months into the pilot it is worthwhile stopping to reflect on what we have learned so far.

Te pilot has highlighted the fact that the legal profession currently lags somewhere behind other similar professions in the introduction of reflective practice. The discussions at the December 2004 conference displayed a lack of a basic understanding of either the processes of reflection or the core elements of portfolios. This was reinforced by the types of questions asked at the initial workshop and during the one to one interviews with legally qualified participants (as opposed to human resources professionals).

This lag has not only led to the need for the initial awareness raising stage but has also led to a disappointing success rate in the numbers participating in the pilot and their level of participation. Even where the main point of contact was both informed and committed, he or she experienced considerable internal resistance from the firm’s partners (and trainees). All of this leads us to observe that there is a great deal that needs to be done by the Law Society of Scotland if portfolios are to be introduced at all. It is also important to overcome partner suspicion and/or resistance for our pilot to be able to provide substantive business and professional benefits.

From the initial feedback we have received from the training firms, and bearing in mind the measures of success outlined earlier, training managers and partners are pleased at the improvement in trainees’ attitudes to their quarterly reviews, which they seem to be taking more ownership over.

The feedback from the students involved in the Diploma pilot shows that many are writing in a reflective way in their personal logs and are better able to recognise their learning from particular episodes, projects and formal skill assessments in a more holistic way. However, we have needed to provide additional support, especially with reflective writing and the identification of tangible evidence.

Our students have already identified some personal benefits. Those yet to find traineeships feel that completing the values questionnaire that formed part of our initial session with them will provide them with confidence to discuss this aspect during any interview selection process. However, they found the tick box approach of the skills self assessment less useful, as this approach encouraged a superficial response. (This confirms the views of the employer firms of the current Law Society tick box logs). All made an attempt to complete the second stage 500 word personal statement. Many found matching this with tangible evidence difficult, and have needed encouragement to recognise opportunities for empirical learning and deep reflection.

Reflecting for ourselves, although at an early stage in the project the signs are encouraging. We feel that we underestimated the lack of knowledge that exists in the legal profession in Scotland about the use of reflection as a professional and personal development tool. We are encouraged, however, by the responses on our Diploma project, where our one to one support is providing direct benefits to our students as indicated above, and for ourselves in the shape of positive reinforcement of our initial aspirations. We are aware of the need to continue to monitor and review over the remainder of the project, and feel confident that the results will provide useful information to inform the legal profession in Scotland on the use of portfolios in practice.

References and bibliography

• Baume D (2001) A briefing on the assessment of portfolios (LTSN Generic Centre Assessment Series No 6) York: LTSN
• Beatty L (1997) Developing your teaching through reflective practice Birmingham: SEDA
• Claxton G (1999) Wise up: the challenge of lifelong learning London: Bloomsbury
• GMC (2001) Good medical practice (3rd ed) London: General Medical Council
• Hinett K (2002) Developing reflective practice in legal education Coventry: UKCLE
• Le Brun M and Johnstone R (1994) The quiet revolution: improving student learning in law Sydney: Law Book Company
• Maharg P (2001) ‘Negotiating the Web: legal skills learning in a virtual community’ International Review of Law Computers and Technology 15(3): 345-361
• Maharg P (2004a) ‘Virtual communities on the Web: transactional learning and teaching’ in A Vedder (ed) Aan het werk met ICT in het academisch onderwijs Nijmegen: Wolf Legal Publishers 74-90
• Maharg P (2004b) ‘Authenticity in learning: transactional learning in virtual communities‘ (Cyberlaws conference paper)
• Maughan C and Webb J (2005) Lawyering skills and the legal process (2nd ed) Cambridge: Cambridge University Press
• McLoughlin C and Luca J (2002) ‘The learner centred approach to developing team skills through Web-based learning and assessment’ British Journal of Educational Technology 33(5): 571 -582
• Moore W (1970) The professions: roles and rules London: Sage
• Paterson A (1996) ‘Professionalism and the legal services market’ International Journal of the Legal Profession 3,(1/2): 137-168
• Paterson A (2000) ‘Self regulation and the future of the profession’ in D Hayton (ed) Law’s future Oxford: Hart
• RCGP (2002) Good medical practice for general practitioners London: Royal College of General Practitioners
• Schon D (1983) The reflective practitioner San Francisco: Jossey-Bass
• Schon D (1987) Educating the reflective practitioner San Francisco: Jossey-Bass
• Svensson L (1990) ‘Knowledge as a professional resource: case studies of architects and psychologists at work’ in R Torstendahl and M Burrage (eds) The formation of the professions: knowledge state and strategy London: Sage
• Webb J (2005) Portfolio-based learning and assessment (teaching resource note) Coventry: UKCLE
• Westwood F (2001) Achieving best practice: shaping professionals for success Maidenhead: McGraw-Hill
• Westwood F (2003) Report to the Law Society of Scotland on the format and content of the Diploma in Legal Practice
• Westwood F (2004) Accelerated best practice: implementing success in professional firms Basingstoke: Palgrave Macmillan

Last Modified: 4 June 2010